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ABSTRACT 

Workshops, conferences, climes, seminars, and 
training sessions are examples of "temporary systems," in which 
participants suspend their usual roles and responsibilities and 
concentrate on a few short-term objectives. One mechanism of a 
temporary system is the use of feedback to determine the extent to 
v/hich the project objectives are being achieved and the impact that 
the project is having on the participants. Typically, two evaluation 
methodologies are used: (1) formative or process, and (2) summative. 
This paper describes five evaluation processes that provide an 
evaluation model for- staff development programs. The first nodel — of 
a formative and process evaluation — is designed to help in the 
Identification of the type of activities that a staff development 
director might plan. Model two illustrates a more summatively 
oriented mstrumenic for judging which selected preconference 
mailings, brochures, etc. were perceived to be most helpful to the 
participant. The third model illustrates an easy to use form after 
each major presentation. Model four is one example of "testing" any 
sel ected process element of a workshop or a longitudinal mservice 
education project. The fifth model provides an opportunity for 
participants to evaluate their perceptions of the over-all 
experiences offered by the staff development program. (JD) 
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Workshops, conferences, clinics, seminars and training 
sessions are all part of the typical educator's yearly repertoire 
of activities. In these sessions, a small number of persons meet 
for a defined period of time, to achieve specified or unspecified 
goals or objectives. Most of these activities would be 
classified by Matthew B. Miles (1964) as being "temporary 
systems," in contrast to the parent organizations or "permanent 
systems . " 

Regarding Temporary Systems 

While participating in a temporary system, participants 
suspend their usual roles and responsibilities from their 
permanent system and concentrate on a few short-term objectives. 
Further, they all know that they will be in the temporary system 
^ for only a short period of time. It is during the temporary 
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system phase of an in-service training project that participants 
are free to try out new ideas, practice a technique without the 
penalty of being wrong, or work in a generally supportive and 
noncompetitive climate, and be away from back-home interruptions. 

To install the elements of temporary systems into the 
evaluation paradigm of staff development programs, there are five 
rather simple phases which need to be accomplished. These are: 
(1; planning or preparing for the project; (2) organizing for the 
project's "start-up"; (3) operating the project; (4) closing the 
system, i.e., preparing the participants for their "back-home" 
roles, and (5) implementing the newly learned skills. 

Temporary systems exist and operate for only short 
durations, have well established goals, anS even have the 
expectancy to cease existence in short periods of time. The ad 
hoc committee is an appropriate example of a temporary system. 
It is established with a definite purpose, has a predictable 
life-span, and no one moans the loss of the Timittee when it is 
dissolved. Yet, such a structure is a vital Ix'ik in helping 
permanent organizations to be innovative and to make needed 
changes for survival. 

Contrasting Permanent and Temporary Systems 

Pernianent organizations tend to have established hierarchies 
or lines of power and autiorlty. Roles, norms, and expectations 
tend to be rather well defined. in the activities of permanent 
systems, participants are often prescribed by position (role) 
rather than by an atmosphere which encourages interaction. 



A temporary system, cont"^*ariwise, provides an environment to 
be creative, innovative, open, trusting and moro communicative 
with all. Often emanating from a temporary system is a product 
or process which can be adapted in the permanent system. As 
these products or processes are being formed, members of the 
temporary system are urged to be innovative or creative. In a 
temporary system, personal needs are met by the ability to 
contribute to the success of the temporary system. There is also 
a tendency for members to become more supportive of each other 
and thus reduce personal def ensiveness . 

The lack of innovativeness in permanent systems may often be 
explained in that individuals expend their energy in trying to 
maintain the existing role-defined relationships within that 
permanent structure of operation. Previously established formal 
lines of communications, and somewhat rigid patterns for 
interactions do not create an atmosphere that is conducive to 
change . 

Most of the temporary systems that take place in our society 
tend to be centered around conventions, trade shows, conferences, 
workshops, retreats, seminars, clinics or short courses. These 
systems all have the common traits of: (1) fixed length of time, 
(2) commonness of goal, (3) homogeneous group identity, and 
(4) site of action away from the permanent organizations. From 
the contacts made at these meetings there is an opportunity to 
meet people back in the permanent system in a more informal 
setting. Such a "network" aids in altering existing 



relationships within a group, and provides one method by which 
quicker entry into the permanent system is expedited- 
The Concept of Feedback 

One mechanism of a temporary system is the us*^ of feedback 
to determine the extent to which the project objectives are being 
achieved and the impact that the project is having on the 
participants. Typically, two evaluation methodologies are used: 
(1) formative or process, and (2) summative. 

Formati ve or process evaluation . Formative evaluation is 
designed to provide feedback in a rather immediate sense. 
Formative instruments are specifically designed to monitor 
selected aspects of the staff development project so that if any 
problems are emerging, they will be quickly identified and 
rectified. For example, if some met..odology is being used which 
irks someone, it will be through formative instruments that such 
trouble-shooting may take place. Participants tend not to inform 
a workshop director about problems or personal concerns until the 
conclusion of the project — which is too late. By asking "stem" 
questions which might identify problems, project directors 
initiate formative evaluation techniques. 

It was this model that has been extensively followed by the 
senior author. However, the "summative" aspects of the 
evaluation paradigm will also be discussed in this paper. 

One example of a formative and process evaluation is 
illustrated in Model Number One. This evaluation form would help 
to identify the type of activities that a staff development 
director might plan. Model One is not a "learning styles" 



checklist, tout a preference listing to determine active and 
passive modes for the participants* 

Model Two illustrates a more summatively-oriented 
instrument. In most workshops or intensive in-service sessions, 
some prior arrangements are made- The manner in which selected 
preconference mailings, brochures, or other materials were 
perceived by the participants is most helpful in the planning of 
future projects. 

Obviously, the authors subscribe totally to the thesis that 
staff developm ent is a process, not an event . One collects 
"soft" data to perfect techniques that work best for adult 
lear^^ofs. 

Model Three illustrates an easy to use form after each major 
presentation- For example, Orlich has used these forms for one, 
two, and three week projects where one summative evaluation would 
not yield accurate or timely information for other presenters- 
Note in Model Three "item G:" This item quickly provides data 
relating to redundancy or innovativeness of workshop topics - 

Model Four is one example of many used in our workshops to 
"test" any selected process element of a workshop or a 
longitudinal in-service education project- The items used on a 
feedback instrument similar to Model Four may be rewritten by a 
project director to reflect participant views- Model Four cjuld 
be given to any group at the end of the first major set of 
activities, such as the end of the first one-half day of a one 
day session- The project director determines the type of 
feedback to be generated by analyzing or speculating about 
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potential trouble areas. Sometimes these trouble areas are a 
most inconsequential matter, e.g., the coffee is too cold, the 
donuts are too filling. Any minor excuse is used by workshops 
attendees to rationalize their own apathy. By using a temporary 
system instrument, these little nuances are quickly rendered to 
the "ineffectual heaps." 

The information gained by using formative evaluation 
instruments should be tabulated immediately and summarized. The 
data must be shared with all the staff so the corrective actions 
or adjustments may be made in those areas which are not being 
received well by the participants. We mentioned earlier that 
feedback was provided to all who were associated with the 
temporary system. The participants are also give a report on 
each of their evaluations! After all, we want them to share in 
the system's decision-making and know how they, as a group, are 
reacting. Where possible, return the rating sheets to the group 
so that each member can locate his or her own and then compare 
the individucl rating sheet with that of the entire group. (This 
can be done by having each member place a code or symbol on the 
instruments) . 

By using this technique there is a strong tendency for 
dissidents to show behavior changes and fall more in line with 
the group. This is especially helpful when one or two persons 
who are "grousing" tend to become disruptive and provide a strong 
negative influence against the project's intended outcomes. But, 
by using temporary systems management strategies, the minority 
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will be shown as just that* Under such cases the majority then 
tends to influence and adjust inappropriate minority behaviors. 

This is not to imply that genuine disagreements are not 
tolerated. Contrariwise, when using these techniques it will be 
readily apparent that conflicts are more easily reconciled. 
Participants have some method of participating in the decision- 
making activities of the project. For those temporary systems 
which are scheduled for 5 to 20 days, it is essential to appoint 
various committees to provide feedback and reports to all. Such 
committees might be: a social committee, evaluation committee, 
publicity committee, grievance committee. The essential element 
when using temporary systems management is that the committees 
are genuinely integrated into the operations. 

The rationale for feedback or formative evaluation is to 
provide data on which to make correctives — immediately, if rot 
sooner! When participants realize that their influence is 
guiding the system, they tend to become more responsible and 
sensitive to all involved in the system. The instructional 
climate and total environment becomes positive and supportive. 
That is precisely the kind of learning climate that one ought to 
subscribe always when conducting any in-service workshop or 
conference . 

Organizing f or"Start-Up" 

The initial perceptions of workshop participants tend to be 
the lasting ones. If you accept the previous statement as being 
a valid generalization, then you will agree that any temporary 
system must get off to a good start. It must be understood that 
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participants make role transitions when attending any program. 
You can aid in this transition by providing some means by which 
staff and participants may become acquainted. A short reception 
the evening before a project begins is one easy method to 
implement. Any type of informal reception or gathering helps to 
reduce the anxiety or feeling of "loneliness" associated with 
cor.ference or seminar attendance. Name tags, short 
introductions, conversations c.nd a "punch bowl" are always sure 
ways to build a strong start-up. 

Depending on length of the temporary system, the director 
might provide a blank list on which the participants list their 
interests in recreational activities, e.g., bridge, golf, hiking, 
swimming and the like. Such lists help the participants to 
locate others with similar interest; and again, this technique 
improves the climate of the temporary system. A short course or 
seminar just doesn't have the time for the typical "gamesmanship" 
that is so obvious in permanent systems. Efficient use of time 
is of essence in a temporary system. Project directors must be 
cognizant of every possible means to institute immediate and 
positive interactions among all who are involved in the temporary 
system. 

Start-up Macro Plan 

If a general plan was not distributed to the participants in 
the preconference materials, then a general or "Macro Plan" 
should be provided at the first formal session. This helps all 
persons to understand what will take place and when. Such a plan 



also "answers" many questions in advance that participants may 
have had, thus saving initial time and effort. 

It should be explained that the participants will have 
opportunities, NB: plural, opportunities; to react on the 
project's conduct. This alone will cause people to be more 
responsible in their own actions. 

All necessary materials, books, paper, pencils, and 
equipment should be ready for the start-up. The project director 
must prepare a written inventory of what is needed so that 
nothing is left to chance. All management activities are focused 
on making the projev-:t meaningful to thor.e attending. 

If small groups are to be established, then a general plan 
is devised so that participants and tasks may be assigned in an 
efficient manner. A varied set of experiences should be provided 
for the start-up. if a guest speaker is to kick-off the project, 
then be certain that the speaker is dynamic and positive. 
Nothing helps to launch a project like a "perceived winner" at 
the start. 

In some cases, the project might be the first of several 
such seminars, conferences or short courses. For example, state 
agency personnel often conduct workc^hops in several different 
locales. If this is so, then after the first project start-up, 
request explicit feedback about the preconf erence activities. 
This feedback will help to improve future conferences. Model 
Two, is an example of such an instrument. 
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Project Operations 

It may be assumed that in most workshops, seminars or short 
courses, there is a mix of lecture, activity, hands-on 
experiences, laboratory work, study of print or nonprint 
materials and the like. Often there will be individual tasks, 
small group tasks or even large group tasks. In more 
comprehensive projects, there may be several different visiting 
"fire persons" who provide instruction. With participants 
functioning in many different roles and learning situations, it 
becomes important for them to provide feedback to the project 
staff on how they are perceiving the temporary system and how 
they are reacting to it. Thus, during the project operations 
chere should be a consistent flow of collected feedback froir. 
those who attend. There is much, much more related to project 
operations, but that is a topic for another time. 
System Close-down 

The start-up activities of a temporary system are aimed at 
making a smooth transition from the permanent system to the 
temporary one. The system close-down is aimed at helping the 
participants finalize plans so they may use the information, 
skills or whatever back in their respective permanent systems. 
Presentations which are specific to orientations on 
implementation procedures should be a part of the system close- 
down. 

Depending on the type of in-service education project, the 
basic mechanism to be used will resemble planning or problem 
sv^lving techniques. In short, the project director helps each 



participant to work out appropriate »»back home" implementation 
plans. 

Further, the final or summative evaluation of the temporary 
system must be conducted. Model Five illustrates one way to 
accomplish that. (Model Five will be discussed l^icer) . 

While there is no way "to closed-down a tempor;^ry system 
gracefully" there should be some means established whereby 
members may contact each other if they so desire. A network 
might even be established for the sharing of information which 
may be of interest to all or some of the participants in the n^ar 
future. The project director should alert each member of the 
temporary system to the possibility of f oil*- /-up and what is 
planned by the leader. This leads to the final phase, the 
follow-up by the director and implementation by the participants. 
Implementation and follow-up 

The primary goal of temporary systems management is to 
maximize the training that is being conducted. Thus, the "back 
nome" activities are focused on implementing the newly learned 
skills. The participants must be certain to present their 
implementation plans to those administrators who must endorse 
them. These include the superintendent of schools, the 
curriculum director and the school principal. r^inally, the 
skills or newly learned activities become a part of the 
curriculum — that is what implementation is all about! 

The project director should distribute news releases about 
those who attended the seminar to the respective local media. 
This is essential so that participants may gain professional 



recognition. Further, the project director may send out short 
questionnaires to determine the degree of implementation over an 
extended period of time. Then there is the preparation of the 
final report. The final report usually contains the basic 
elements of goals, objectives, procedures, and evaluation data. 
The collected data, in the formative processes and summative 
evaliiations, are synthesized for the final report document. Time 
permitting, follow-up data might be added or it maybe provided as 
a future addendum. 

Temporary systems management is one technique that can be 
used to facilitate in-service training. By emphasizing feedback, 
there is focus on decision-making processes and .mmediate 
adjustment needed to established and maintain a responsive 
environment for the pai cicipants . Evaluation is used as a 
continuous process rather than as a one-shot task which never 
impacts the training activities. The use of temporary systems 
management does cause more work for the project director. But, 
properly used temporary systems management also means a 
successful project — from all perspectives. 
Problems of Temporarv Systems 

The planned incorporation of a temporary system into a 
permanent system requires that the two structures oe supported 
simultaneously. The permanent structures, of course, keep the 
organization intact. Temporary systems emerge as needed to cope 
with the stresses whicn are not reconciled in the permanent 
organization. 



There are at least five problems which may be anticipated 
when using temporary systems* 

1* There may be too much stimulus in the temporary system 
causing tho participants to over exert themselves as they manage 
their extra work loads. 

2. The liberation from the permanent organization may 
generate unrealistic goals or expectations which cannot be 
attained. 

3. Many individuals will initially lack the interactive 
skills which are necessary to cope with others in an intimate, 
and intensive interpersonal climate. 

4. A serse of alienation can develop as persons may 
perceive the possibility of producing changes in the permanent 
organization as being too remote. 

5. Because innovations may take time to be implemented 
fully, they may not obtain support from the permanent structure. 
This situation then causes a linkage break-down between the 
systems. 

None of the above five is an insurmountable problem, but 
each should be anticipated and a plan developed to deal with it. 

Summative Perceptions 

ThosCi final evaluations of a project or workshop are 
completed to give a glimpse of how people "feel" when they leave. 
Key the summative evaluation elements to the major objectives of 
the project. Model Five illustrates one that we nave used since 
1972. One tabulates these data to determine long-range trends. 



Forms such as these are charted to show the relative "perceived 
effectiveness" by various participants. 
Multi-year Field Test of the Model 

One major objective of this paper is to report the findings 
of a series of studies designed to determine the efficacy of 
using formative (interactive or transactional) techniques to 
evaluate 17 different intensive workshops directed by the senior 
author between 1972 and 1989. 

Subjects . The subjects were participants in a series of 
different National Science Foundation projects conducted during 
the summers of 1972, 1973, 1974, 1975, 1978, 1979, 1980, 1981, 
1982, and three separate workshops held in 1986. In 1988, three 
one-week workshops were conducted, and two one-week workshops in 
1984. Each group was comprised of 35, 45, 28, 50, 51, 46, 45, 
44, 48, 25, 12, and 40 different individuals, respectively. In 
1988, groups were 24, 25, 25. In 1989 groups were 20 and 25. 
The subjects were elementary or middle school teachers, 
principals, intermediate school district curriculum personnel, 
administrators, or science consultants. All subjects attended 
three-week instructional programs of 1972, 19 74 and 1978 as 
volunteers. In 1975, the conference was six days and in 1978 was 
eight. The 1980, 1981, 1982 and 1986, 1988 and 1989 c erences 
were five days in length. (All persons received expenses and 
free tuition) . 

Procedures . To accomplish the formative evaluations, a 
series of instruments were administered to determine the 
"environment" of the project as perceived by the participants. 
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The instruiaents were designed in part from an institute conducted 
in 1972 by Robert Bernoff, Emily Girault, Mary Budd Rowe, Matthew 
B. Miles and Dale Lake (1973) and adapted by D.C. Oriich (1978) • 
Especially important to the conceptual basis of this study was 
the '-Tempcrary Systems" theory proposed by Miles (1964) . This 
evaluation model is also featured in Designing Shorr-Term 
Instructional Programs (Waterman et al., 1979) and Staff 
Development: Enhancing Human Potential (Oriich, 1989) . Implicit 
to the evaluation system is the assumption that "affective" 
feedback is desirable by instructional managers or workshop 
directors. Frequent evaluations were administered during the 
conduct of the projects, ranging from daily to weekly. 

Model Five shows one of the final evaluation forms 
administered on the last day of each project. Thus, comparable 
data were "luckily'' collected over a 17 year history. 

Observe Table 1 which shows the mean responses for the eight 
categories which are shown on Model Five. Using a zero to five 
scale, there were only nine times in 96 categories that the mean 
evaluation fell below 4.0 (5.0 is the absolute top). Even less 
variation would be shown by the group medians which were ranked 
at 5.0 almost consistently. Further observe the consistency of 
particip.-^nt responses for the 17 different workshops. 

Tables 2, 3 and 4 and 5 show the same data but arranged by 
blocks. Those are shown here so that staff development directors 
might plan for such a system by which data are collected to 
provide evidence regarding the relative effectiveness of 
different workshops or their presenters. 



Figures 1, 2, 3, 4, 5, 6 and 7 are provided to illustrate 
the impact of the evaluations in graphic form. 

Conclusions and Implications 

The techniques and instruments in our projects have provided 
one means by which to measure the direction of participant 
affectivity and perceptions of non-cognitive dimensions of 
instruction which may affect cognitive achievement. 

The technique and instruments require an open system of 
interaction between workshop or instructional leaders and 
learners. Further, the installation of a formative or 
interactive evaluation system requires systematic and frequent 
monitoring of participants and the utilization of the information 
by the leaders thereby generating greater group cohesiveness, 
solidarity, and program adjustment. The latter did take place in 
all special projects sponsored by NSF at Washington state 
University, 

The use of formative data may be used as one evaluative 
technique by which to judge or evaluate: (1) instructors, 
(2) instructional techniques, (3) group responsiveness to program 
management, (4) general affective environment of intensive 
learning experiences, (5) workshop or short-term science 
education projects, and (6) trends. Used in conjunction with 
other evaluative techniques, a rather "realistic" evaluation 
pattern will emerge. Ideally, these data will then be used to 
adjust, modify or drop specific staff development activities. 

References 



Robert Bernoff, Emily Girault and Mary Budd Rowe. The 

Application of Temporary System Concepts To Effective 
Planning and Management of Short Term Educationa l 
Programs , Report of grant project G-4508 to the 
National Science Foundation, c. 1973, 68 pp. 

Linda Dubaldi, George Dawson, Robert Bernoff, Donald C. 
Orlich and David R. Stronck. Designs for 
Implementation . Morristown, New Jersey: Silver 
Burdett, 1975. 

Matthew B. Miles, "On Temporary Systems." In Matthew B. 
Miles (Ed.), Innovations in Education , New york: 
Teachers College Press, 1964, pp. 437-486. 

Donald C. Orlich, "Conducting and Evaluating Temporary 

Systems in Vocational Education." In Donald C. Orlich, 
et al.. Evaluation Models for Vocational Educators , 
Olympia: State of Washington Commission for Vocational 
Education, Research Coordinating Unit, April 1978, 
pp. 33-70. 

Donald C. Orlich, Staff Development; Enhancing Human Potential . 
Boston: Allyn and Bacon, 1989. 

Floyd T. Waterman, Theodore E. Andrews, W. Robert Houston, 

Brenda L. Bryant, and Roger Penkratz. Designing Short- 
Term In structional Programs . Washington, D.C.: Association 
of Teacher Educators, 1979. 



The projects reported in this paper from 1972 through 1986 
were all funded by the Division of Pre-College Education in 
Science of the National Science Foundation. Specific programs 
included: (1) Cooperative College-School Science Program (CC- 
SS) , (2) Instructional Improvement Implementation Program (III), 

(3) Information Dissemination for Science Education (IDSE) , and 

(4) Precollege Science and Mathematics Education; Local and 
Regional Teacher Development. 

Portions of this material are based upon work supported by 
the National Science Foundation under Grant No. TEI--8470642 - Any 
opinions, findings, and conclusions or recommendations expressed 
in this publicarion are those of the author and do not 
necessarily reflect the views of the National Science Foundation. 



18 



The projects reported for 1988 and 1989 were funded through 
Title II, Higher Education Act of The Education Economic Security 
Act of 1984. Findings reported here are of the authors and do 
not necessarily reflect the views of the granting agency. 



19 



Model One 

FiiRCEPTIONS ON COURSE/WORKSHOP CONDUCT 



Please circle one number on the scale at a point which indicates 
the direction of your preferred structure for the general conduct 
of this class/workshop. 



3. 



4. 



7. 



Individualized 
activities or 
projects 

Active roles 
for participants 



The instructor to 
present all 
information 

Mutual assistance 
and sharing 
information 

General, 
unthreatening 
feedback only 

Lectures and 
some general 
class 

discussions 



Evaluation (Grade) 
based on some 
prescribed 
criteria o 



I PREFER 



1 2 



1 2 



1 2 



1 2 



1 2 



1 2 



4 5 



4 5 



4 _5 



Group 
activities 
or projects 

Receptive 
role for 
participants 

Group members 
to share reports 
or research 

Separate 
individual work 



Specific, candid 
professional 
judgments 

A mix of lecture, 
small group 
discussions & 
individual 
conferences 

Evaluation (grade) 
based on 
consensus 



1 2 



8 . Comments : 
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Model Two 



FEEDBACK ON PRECONFERENCE MATERIALS 

Directions; Please evaluate each statement by placing 
the code below in the space provided. 



3 
2 
1 
NC 



Very helpful 
Helped somewhat 
Unnecessary 

Not included in my packet 



_1. The maps, travel information, lodging, needed 
clothing and equipment list. 

_2. The list of guest presenters. 

_3. The general project schedule. 

_4. The list of kno//n participants. 

5. Recreational hints and activities available at 
the conference site. 

6. The registration forms. 

7. The original announcement about the 
conference. 



Please add any other items of information that you 
would like to have known in advance. Explain any item 
that you ranked lower than "3". 
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Hodel Three 

CONTENT EVALUATION/PRESENTER CRITIQUE 

Place the number that best describes your reactions to each content 
presenation on the line in front of each item. Use the following code"; 

1. NOT WORTHWHILE 

2. OF LITTLE WORTH 

3 . WORTHWHILE 

4. VERY WORTHWHILE 

Content Topic (or presenter) 

^A. The introduction 

B. The print materials 

C . The discussion 

The participation in selected activities 



_E. The overall presentation 



_^F. Your reaction to total presentation 



How much of the content did you know before this presentation: 
(Check only one of the following responses) 

1. I was very knowledgeable. 



2. I had previous exposure. 

3- I only knew that there was such consent, but 

knew very little about it. 

^- ^ knew absolutely nothing of the content. This 

was my first formal experience with it. 

Add one word that best describes your reaction. 

H. The presentation was: 

I. Comments: 
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Model Four 
QUICKIE FEEDBJ^CK 



Direction s; Place an X on each line above the category 
which best describes your reactions. 

1. How has the workshop progressed to date? 



Moving very slowly* Could move faster Moving along nicely 



2. How is your participation in the workshop? 



Not with it at all* Could participate more Really with it 



3. Is the organization and conduce of the workshop meetinq 
your expectations? ^ 



Not at all* Meeting some, not others* Meeting them well 



4. Was the preconf erence information adequate for you to 
make an easy adjustment of translating the content from the 
pro:)ect to your class? 



Very inadequate* Inadequate* About right Very adequati 
*Comments or suggestions: 



2 o 
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Model Five 



CONFERENCE SUMMATIVE PERCEPTIONS 

^ We expect that you may be having a variety of experiences in this 
project and, of course, these experiences affect what you will learn (or 
nave already learned) . These experiences and consequent learnings will aic 
the director of the project to improve the project. 

Carefully read through the items below. For each one, circle one 
number on the line showing how well you think the various management tasks 
have been done by the director and the staff. 



1. Project goals were 
not specified clearly. 

2. Climate of this 
project was poor. 

3. The "wrong" people came 
to this project. 

4. Overall design of this 
project was quite 
ineffective. 

5. Project did not get 
off to a good start. 

6. This project will 
have no influence 
on how I approach 
science curricula. 

7. Staff resources were 
poorly used in this 
project. 

8. There have been no 
"experiential" or 
"hands-on" learning 
activities used in 
this project. 



FREQUENCIES 

0 1 2 3 4 5 

0 1 2 3 4 5 

0 1 2 3 4 5 

0 1 2 3 4 5 

0 1 2 3 4 5 

0 1 2 3 4 5 

0 1 2 3 4 5 



0 1 2 3 4 5 



Project goals were 
specified very 
clearly. 

Climate of this 
project was very good. 

The "right" people 
came to this project. 

Overall design of this 
project was quite 
effective. 

Project got off to 
a very good start. 

This project will 
strongly influence ho^ 
I approach science 
curricula. 

Staff resources were 
well used in this 
project. 

"Experiential" or 
"hands-on" activities 
ha^^e been frequently 
used in this project. 
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Table 1. 



MEAN RESPONSE SCORES FROM 197 2 TO 1989 



Year 



1972 1973 1974 1975 1978 1979 1980 1981 1982 1986 1988 1989 Mean 



Goals 
Climate 



4.4 
4.9 



3 . 6 
4.2 



4 . 2 
4.5 



3.7 
4.5 



4.0 
4.5 



4.0 
4.1 



4 . 2 
4.5 



4 . 0 
4 . 6 



4 . 0 
4 . 4 



4.2 
4.8 



4.7 
4 . 9 



4 . 6 
4.9 



4.1 

4 . 6 



People 
Design 



4.6 
4.5 



4.1 
3.7 



3 . 3 

4 . 0 



4.6 
4.3 



4.0 
4.1 



3.8 
4 .2 



4 . 4 
4 . 6 



3.8 
4.3 



4.0 
4.2 



4.5 
4 . 4 



4.8 
4.7 



4 . 9 
4.7 



4 . 2 
4 . 3 



Start 
Influence 



4 . 4 
4.8 



4.2 
4.8 



4 . 0 
4 . 2 



4.7 
4.3 



4 .4 
4 .4 



4.1 
4.3 



4.7 
4 . 4 



4 . 3 
4 . 0 



4 . 4 
3.9 



4 . 6 
4 . 2 



4.8 
4.8 



4.8 
4 . 6 



4.5 
4 . 4 



Staff 
Hands-on 



4 . 6 
4.8 



3.5 
4.8 



4 . 3 
4 . 3 



3.9 
4 . 4 



4 .2 
4.6 



4.1 
3.9 



4 . 3 
4 . 4 



4 . 1 
4.7 



4 . 3 
4 . 6 



4.5 
4 . 7 



4.8 



4.8 
5.0 



4 . 3 
4.6 
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Table 2. 



MEAN REPSONSE SCORES FROM 197 2 TO 197 5 



Year 



1972 



1973 



1974 



1975 



Mean 



Goals 
Climate 



-' . 4 
4 . 9 



3 . 6 
4.2 



2 
5 



3.7 
4.5 



4.0 
4.5 



People 
Design 



4 . 6 
4.5 



4 
3 



3, 
4, 



4 . 6 
4.3 



4 . 1 
4 . 1 



Start 
Influence 



4 . 4 
4 . 8 



4 . 0 
4 . 2 



4.7 
4 . 3 



4 . 3 
4 .b 



Staff 
Hands-on 



4 . 6 
4 . 8 



,5 
, 8 



4.3 
4.3 



3.9 
4 . 4 



4 . 1 
4 . 6 



Table 3. 



MEAN REPSONSE SCORES FROM 1978 TO 1982 



Year 



1978 



1979 



1980 



1981 



1982 



Mean 



Goals 
Climate 



4 . 0 
4 . 5 



4.0 
4 . 1 



0 
6 



4 . 0 
4.4 



4 
4 



People 
Desian 



4 . 0 
4 . 1 



3 
4 



0 
3 



Start 
Influence 



4.4 
4 . 4 



3 
0 



4 
4 



Staff 
Hands-on 



4 . 2 
4 . 6 



4.3 
4.4 



4 . 1 
4.7 



4 . 2 
4 . 4 
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Table 4. 



MEAN REPSONSE SCORES FROM 1986 TO 1989 



Year 



1986 



1988 



1989a 



1989b 



Mean 



Goals 
Climate 



4.2 
4.8 



4.7 
4.9 



4.7 
4.9 



4.5 
4 . 8 



4.5 
4.9 



People 
Design 



4, 
4 , 



4.8 
4.8 



4.9 
4.9 



4.9 
4.7 



Start 
Influence 



4.9 
4.8 



4.9 
4 . ^ 



4.8 
4 . 7 



Staff 
Hands-on 



5 
7 



4.9 



4.8 
5.0 



4.8 
5.0 



4, 
4 , 



Table 5. 



MEAN REPSONSE SCORES FOR THREE PROGRAI-IS IN 1988 



Year 



1988a 



1988b 1988c 



Mean 



Goals 
Climate 



4.7 
4.9 



4.6 
4.9 



4.8 
4.9 



4.7 
4.9 



People 
Design 



4.9 
4.8 



4.8 
4.7 



4.9 
4.9 



4.8 
4.8 



Start 
Influence 



4.9 
4.9 



4.8 
4.8 



4.9 
4.9 



4 . 9 
4.8 



Staff 



5.0 



4.8 



4.9 



4 . 9 
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FIG. 1 MEAN RESPONSE SCORES FOR GOALS, 
CLIMATE, PEOPLE, AND DESIGN FOR 1972-1982 
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FIG. 2 MEAN RESPONSE SCORES FOR START UP, 
INFLUENCE ON TEACHING, USE OF STAFF, AND 
HANDS-ON ACTIUITIES FOR 1972-1982 
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FIG. 3 MEAN RESPONSE SCORES FOR PROGRAMS 
IN 1986, 1988, AND 1989 
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FIG. 4 MEAN RESPONSE SCORES FOR THREE 
PROGRAMS IN 1988 
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5 MEAN RESPONSE SCORES FOR TWO 
PROGRAMS IN 1989 
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FIG. 6 MEAN RESPONSE SCORES GROUPED BV 
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FIG. 7 MEAN RESPONSE SCORES FOR THREE 
PRC ;RAMS in 1986 
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